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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission dated January 
23, 2009 has been entered. 

Response to Amendment 

2. Claims 1 and 4-21 remain pending in the application. No claims are currently 
amended. Claims 2 and 3 remain canceled. No new claims are added. 

Response to Arguments 

3. With regard to the applicant's remarks dated December 29, 2008: 
regarding the rejection of claims 1,11, and 21 under 35 U.S.C. 1 03(a) as being 

unpatentable over Best, JR et al. in view of Mora, applicants' arguments have been fully 
considered but they are not persuasive. In particular, applicants argue regarding par. 
[0027], [0031], and [0034] of Mora that "neither a received e-mail nor a calendar entry is 
a piece of multimedia information that includes one of audio information, still image 
information, and video information" and that "presence detection in Mora is not 
discussed for changing status as recited in claim 1". Examiner disagrees with 
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applicants' argument that Mora teachings in at least par. [0027], [0031], and [0034] do 
not meet the claim limitations. In particular, Mora clearly shows in at least par. [0027], 
[0031], and [0034] that multimedia information (i.e. motion event information) is used to 
determine status related to mere presence and beyond that, possibly in combination 
with other data, such as from the PIM 124 and manual user input. In particular, Mora 
shows at Fig. 2 that Messenger Assistant 122 communicates directly with the PIM 124. 
A presence detection application 129 passes presence information from presence 
detection events to the Messenger Assistant 122. The Messenger Assistant 122 sends 
data to the PIM 124. Thus, Messenger Assistant 122, the PIM 124 and Presence 
Detection Application 129 are in communication with each other and the status "beyond 
mere presence" is based on at least one or more of the data objects received from the 
PIM 124 (PIM Event 138 at Fig. 3), Presence Detection Application 129 (Presence 
Event 140 at Fig. 3), and User Input 136 at Fig. 3. Therefore, the combination of Best 
and Mora teachings meet the claim limitations and the rejection is maintained. 

Applicants are reminded that the limitation of "the multimedia information 
comprises at least one of audio information, still image information, and video 
information" is open-ended, that is it may include other information such as PIM entries 
and user input in Mora as part of "multimedia information". There is nothing in the claim 
that would limit the multimedia information to at least one of audio, still image and video 
information. In addition, there is nothing in the claim that would require determination of 
user status be based exclusively on the captured multimedia information, wherein the 
multimedia information is limited to audio, still image, video, or a combination of audio, 



Application/Control Number: 10/644,270 Page 4 

Art Unit: 2442 

still image and video information, as discussed above. Such that, determination of the 
user status beyond mere presence can be based on combination of at least one of 
audio, still image and video information and some other data, such as PIM entries and 
user input in Mora. Only if each of independent claims is amended to limit multimedia 
information to recited audio, still image, video information, or a combination thereof, and 
to further specify that status of the user is determined exclusively based on said 
multimedia information can applicants' arguments be held as persuasive. If applicants 
choose to amend the claims by replacing "comprises" with "consisting of, applicants are 
advised to review MPEP section 21 1 1 .03 on transitional phrases to determine how 
transitional phrase "comprising" in the preamble may affect claim scope in combination 
with transitional phrase "consisting of. 

As to any arguments not specifically addressed, they are the same as those 
discussed above. 

Claim Rejections - 35 USC § 101 

4. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

5. Claims 1 and 21 are rejected under 35 U.S.C. 101 because the claimed invention 
is directed to non-statutory subject matter. 

As to claims 1 and 21 , use of the word "system" does not inherently mean that 
the claim is directed to a machine. Only if at least one of the claimed elements of the 
system is a physical part of a system can the system as claimed constitute part of a 
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device or a combination of devices to be a machine within the meaning of 35 U.S.C. 
101. 

In the instant case, the system comprises an information capture module, an 
information extraction and analysis module, an information interpretation module, and 
an Application Program Interface module. The specification fails to provide a specific 
definition of either one of the modules as necessarily including some sort of hardware 
elements. It is noted that the fact that "in one embodiment, the information capture 
module 210 includes the conventional components of a digital camera, which relate to 
the capture and storage of multi-media data" as discussed at par. [0031] of the 
specification, does not necessarily limit the information capture module 210 to include 
conventional components of a digital camera. Thus, it would be reasonable to conclude 
that in at least one other embodiment, the information capture module includes only 
software application for capturing the multimedia information. Applicants are advised to 
amend claims 1 and 21 to further specify that the information capture module includes 
at least one of the hardware components, as discussed at par. [0031]. 

Therefore, claims 1 and 21 are directed to non-statutory subject matter. 



Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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7. Claims 1,4-6,9-12, 14-16, 18, and 21 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Best, JR. et al. (US 2005/0034147 A1) (hereinafter Best) in 
view of Mora (US 2004/0162882 A1). 

As to claim 1 , the preamble has been given patentable weight since the claim 
body refers back to the preamble. See "the IM application", "the user", and "the 
multimedia information" at lines 1-2 of the claim body. 

As to claim 1, Best shows a system [Fig. 2A] for performing an action on an 
Instant Messaging (IM) application [in response to receiving a presence indication] (par. 
[0029]), wherein the performed action is based on multimedia information [presence 
indication], the system comprising: 

an information capture module [presence detector (220)] that is capable of being 
used for capturing the multimedia information in the vicinity of a machine on which the 
user is using the IM application (par. [0009] I. 4-7, par. [0029] I. 21-25), wherein the 
multimedia information comprises at least one of audio information, still image 
information, and video information (par. [0023] I. 15-16); 

an information extraction and analysis module [visual identification logic] that is 
communicatively coupled with the information capture module (par. [0023] I. 14-16) and 
is capable of being used for extracting relevant information from the captured 
multimedia information (par. [0023] I. 13-19); and 

an information interpretation module that is communicatively coupled with the 
information extraction and analysis module [presence detector instructions (216)] that 
are capable of being used for interpreting the extracted and analyzed information for the 
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IM application to determine the presence of a user (par. [0028], par. [0009] I. 4-7), 
wherein the interpreted information is used for directing a launch of the IM application 
(par. [0029] I. 1-11). 

Best does not show that the performed action in response to receiving a 
presence indicator is updating an IM application. In Best, the "performed action" is 
launching an IM application, in one embodiment, par. [0029] I. 1-11, and updating a 
presence database, in another embodiment, par. [0039]. 

Best also does not show determining a status of said user, wherein the 
interpreted information is used for updating the IM application to set forth said status of 
the user, and an Application Program Interface module for the IM application, 
communicatively coupled to the information interpretation module, for receiving the 
interpreted information and updating the IM application regarding the user when 
present, wherein the Application Program Interface module is configured to update the 
user's status on the IM application. 

Mora shows updating an IM application regarding a user of the IM application 
(par. [0024] I. 15-20, par. [0025] I. 1-2 and 18-22), wherein the updating is based on 
multimedia information [user input event, PIM event, motion event] (Fig. 3), wherein the 
multimedia information comprises at least one of audio information, still image 
information, and video information [motion event comprises video information] (par. 
[0022]). 

Mora also shows determining a status of said user (par. [0020] I. 3-7, par. 
[0027]), wherein the interpreted information [availability and presence information] is 
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used for updating the IM application to set forth said status of the user (par. [0031] I. 5- 
17); and 

an Application Program Interface module for the IM application [Messenger 
Assistant] for receiving the interpreted information and updating the IM application 
regarding the user when present [Messenger Assistant interfaces availability and 
presence information with that users' instant messaging capabilities, automatically 
indicates the users' availability state to others in real time] (par. [0020] I. 1-7, par. [0024] 
I. 15-20, par. [0031] I. 5-17), 

wherein the Application Program Interface module is configured to update the 
user's status on the IM application (par. [0024] I. 15-20, par. [0025] I. 18-22, par. [0027] 
I. 16-21, par. [0031] I. 5-17). 

It would have been obvious to one or ordinary skill in the art at the time of the 
invention to modify the system of Best by determining a status of the user and having 
an Application Program Interface module of Mora that is configured to update the user's 
status on the IM application in order to provide business associates with up to date 
personal status on IM application automatically and in real time in response to receiving 
availability and presence information of the user from various sources (par. [0012], 
[0020] in Mora). 

As to claim 4, Best in view of Mora shows that the user's status is updated in a 
buddy window on the IM application (par. [0019], [0031] I. 5-17 in Mora). 
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As to claim 5, Best in view of Mora shows that the user's status comprises at 
least one of available, busy, on the phone, and away from the desk (par. [0025] I. 7-9, 
par. [0026] I. 7-9, par. [0027] I. 16-21 in Mora). 

As to claim 6, Best in view of Mora shows that the Application Program Interface 
module [Message Assistant] (par. [0020] I. 1-7 in Mora) is capable of being configured 
to update the user's identity on the IM application [determining the identity of the user] 
(par. [0023] I. 16-19, par. [0035] in Best). 

As to claims 9 and 14, Best shows employing motion detection techniques for 
extracting relevant information from the captured multimedia information for detecting 
motion (par. [0022] I. 10-24). 

As to claim 10, Best shows employing face recognition techniques for extracting 
relevant information from the captured multimedia information (par. [0023] I. 3-5 and 17- 
19). 

As to claim 1 1 , the preamble has been given patentable weight since the claim 
body refers back to the preamble. See "the captured multimedia information" at line 1 , 
"the IM application" at line 4, and "said user" at line 5 of the claim body. 

As to claim 1 1 , Best shows a method [Fig. 2A] for performing an action on an 
Instant Messaging (IM) application [in response to receiving a presence indication] (par. 
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[0029]) regarding a user based on captured multimedia information [presence 
indication], the method comprising: 

receiving the captured multimedia information (par. [0028] I. 1-5) [wherein the 
multimedia information comprises video images] (par. [0023] I. 15-16), wherein the 
multimedia information comprises at least one of audio information, still image 
information, and video information (par. [0023] I. 15-16); 

extracting and analyzing relevant information from the captured multimedia 
information (par. [0023] I. 13-19); 

interpreting the analyzed information for the IM application [determining whether 
computer should take any action based on at least in part on the received information] 
(par. [0028] I. 5-8) to determine the presence of a user (par. [0028], par. [0009] I. 4-7, 
[wherein the interpreted information is used for directing a launch of the IM application] 
(par. [0029] I. 1-11). 

Best does not show that the performed action in response to receiving a 
presence indicator is updating an IM application. In Best, the "performed action" is 
launching an IM application, in one embodiment, par. [0029] I. 1-11, and updating a 
presence database, in another embodiment, par. [0039]. 

Best also does not show determining a status of said user when present, 
providing the interpreted information to the IM application, and updating the IM 
application based on the provided information to set forth said status of the user. 

Mora shows updating an IM application regarding a user of the IM application 
(par. [0024] I. 15-20, par. [0025] I. 1-2 and 18-22), wherein the updating is based on 
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multimedia information [user input event, PIM event, motion event] (Fig. 3), wherein the 
multimedia information comprises at least one of audio information, still image 
information, and video information [motion event comprises video information] (par. 
[0022]). 

Mora also shows determining a status of said user when present (par. [0020] I. 3- 
7, par. [0027]), providing the interpreted information to the IM application (par. [0031] I. 
5-17), wherein the interpreted information [availability and presence information] is used 
for updating the IM application to set forth said status of the user (par. [0031] I. 5-17). 

It would have been obvious to one or ordinary skill in the art at the time of the 
invention to modify the method of Best by determining a status of the user when 
present, providing the interpreted information to the IM application, and updating the IM 
application based on the provided information to set forth said status of the user in order 
to provide business associates with up to date personal status on IM application 
automatically and in real time in response to receiving availability and presence 
information of the user from various sources (par. [0012], [0020] in Mora). 

As to claim 12, Best in view of Mora shows all the elements, as discussed per 
claim 4, above. 

As to claim 15, Best in view of Mora shows all the elements, as discussed per 
claim 6, above. 
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As to claim 16, Best in view of Mora shows all the elements, as discussed per 
claim 10, above. 

As to claim 18, Best shows that updating is performed only after a user trigger 
[person approaching a computer (210) is detected by the presence detector (220) and 
GUI icon is changed in response to this user trigger] (par. [0028] and [0039]). It is being 
noted that even though Applicants consider "a user trigger" being "a detected gesture of 
the user to activate the status reporting feature" (see Reply filed on July 26, 2007 under 
Claim 18 arguments), no such limitation (gesture of the user) is being claimed, as per 
claim 18. As the result, "a user trigger" is being reasonably interpreted as "person 
approaching a computer" in Best reference. This interpretation is supported by the 
specification wherein it is stated: "... trigger events can include ... a user's approaching 
of the camera ..." (par. [0039] in the current specification). 

As to claim 21 , Best in view of Mora shows all the elements, as discussed per 
claims 1, 4, and 5, wherein claim 21 includes all the elements of claim 1 with addition of: 

wherein the user's status is updated in a buddy window on the IM application (as 
discussed per claim 4); 

wherein the user's status comprises at least one of available, busy, on the 
phone, and away from the desk (as discussed per claim 5). 



Application/Control Number: 10/644,270 Page 13 

Art Unit: 2442 

8. Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Best, JR. 
et al. in view of Mora in further view of Mastrianni et al. (US 2002/01 14519 A1 ). 

As to claim 7, Best in view of Mora shows all the elements except for logging out 
a previous user, and logging in the user on the IM application. 

Mastrianni shows that updating the user's identity comprises logging out a 
previous user, and logging in the user [logging in the user at step (316) Fig. 3 and 
logging out the user at step (320) Fig. 3 wherein the previous user is the user that was 
previously logged in]. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the system of Best in view of Mora by logging in and out the user in 
order to update the user's identity when the user walks away from computing device at 
step (320) in Mastrianni. 

9. Claims 8, 13, and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Best, JR. et al. in view of Mora and in further view of Toyama (US 2006/01 93494 
A1). 

As to claims 8 and 13, Best in view of Mora shows all the elements except for 
employing face tracking techniques for tracking a face. 

Toyama shows employing face tracking techniques for extracting relevant 
information from the captured multimedia information for tracking a face (par. [0034] I. 1- 
3, par. [0035]). 
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It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the system and method of Best in view of Mora by employing face 
tracking techniques for extracting relevant information from the captured multimedia 
information for tracking a face in order to determine if the user is looking at the monitor 
and cease speech recognition if the user is turned away (par. [0035] in Toyama). 

As to claim 19, Best in view of Mora shows all the elements except for the 
captured multimedia information (multimedia data) includes audio data. 

Toyama shows that multimedia data includes audio data [speech] (par. [0035] I. 
11-19). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the method of Best in view of Mora by having the captured 
multimedia information include audio data in order to capture user speaking while facing 
the monitor (Toyama, par. [0035] I. 11-19). 

10. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over Best, 
JR. et al. in view of Mora and in further view of Harris (US 7,202,798 B2). 

As to claim 1 7, Best in view of Mora shows that the status of said user comprises 
whether said user is on the phone (par. [0019] I. 16 in Mora). 

Alternatively, Harris shows that the status of the user comprises whether said 
user is on the phone (col. 5 lines 46-67 and col. 6 lines 1-13) [wherein a miniature 
camera (605) takes a real time image of the user (610) and automatically recognizes 
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portable telephone use. Upon detecting a phone usage, a signal is sent to an attendant 
notifying of phone usage by the user (610)]. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the method of Best in view of Mora by having the status of the user 
comprise whether said user is on the phone in order to notify User B (in Best reference 
or "attendant" in Harris reference) that User A (in Best reference or user (610) in Harris) 
is on the phone (par. [0039] in Best, col. 6 lines 46-67 in Harris). 

1 1 . Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Best, 
JR. et al. in view of Mora and in further view of Johnson et al. (US 5,349,662). 

As to claim 20, Best in view of Mora shows that the status of said user being 
indicated as busy if said user is detected to be engaged in one of the predetermined 
appointment events (par. [0032] I. 1-12, Fig. 4B in Mora). 

Best in view of Mora does not show that indication of status as busy is performed 
if said user is determined to be using a program other than IM or email. 

Johnson shows determining if said user is using a program [a spreadsheet 
program] (col. 6 line 33) other than IM or email [User Activity Event Detection Process] 
(col. 6 lines 1-38, Fig. 2). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the method of Best in view of Mora by having the indication of status 
as busy being performed if said user is determined to be using a program other than IM 
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or email in order to properly indicate the user status corresponding to the user activity 
event such as invocation of a spreadsheet program. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to OLEG SURVILLO whose telephone number is 
(571)272-9691 . The examiner can normally be reached on M-Th 8:30am - 6:00pm; F 
8:30am - 5:00pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Caldwell can be reached on 571-272-3868. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Examiner: Oleg Survillo /Andrew Caldwell/ 

Supervisory Patent Examiner, Art 
Phone: 571 -272-9691 Unit 2442 
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